Programme For International Student Assessment Test Research Articles

A comparative study of critical reading abilities among students in Malaysia and Vietnam: Insights from PISA-based assessment

This study conducted a comparative analysis of critical reading skills among engineering students at private universities in Malaysia and Vietnam, using the Programme for International Student Assessment (PISA) reading framework. Involving 182 undergraduates, the research included a quantitative critical reading test followed by statistical analysis (T-tests) to examine score variability among first-year and final-year students. The analysis revealed significant performance differences among first-year students but not among final-year students from both countries. To complement the quantitative data, semi-structured interviews with 16 participants across different critical reading levels were conducted. These interviews aimed to capture students’ perspectives on the PISA test, highlighting challenges such as the Western orientation of texts, English vocabulary, and the format of open-ended questions. Integrating quantitative and qualitative methods, the study offers an in-depth understanding of the critical reading abilities of engineering students in Malaysia and Vietnam, underscoring the importance of considering cultural and linguistic factors in educational assessments.

How does pre-school attendance affect school performance? An application of Gini-BMA methodology on PISA 2018 dataset

Evidence flourishes in the literature of a direct link between asymmetries in pre-primary education and achievement test scores in the primary school. However, children tend to reap the greatest benefits if school programs are of high quality. Premised on this commonality between pre-primary education and academic achievement in the primary school, and considering variations in quality indicators relative to students’,schools’ and families’ characteristics, this study mainly investigates the effect of universal pre-school education on the students’ academic results in 24 OECD and non-OECD countries. Data are obtained from the OECD Programme for International Student Assessment (PISA) 2018. Employing a Gini Regression Bayesian Model Averaging (BMA) approach to account for theory uncertainty we find that attendance of pre-primary institutions has a beneficial impact on PISA test scores for 15-year-old students in most of the countries. However, there are cases where early childhood coincides with a decline in students’ outcomes in both reading and science.

Evaluating the Results of PISA Assessment: Are There Gaps Between the Teaching of Mathematical Literacy at Schools and in PISA Assessment?

<p style="text-align: justify;">The problems in education in the countries of the Organization for Economic Cooperation and Development (OECD) vary from country to country. The differences between "upper class" and "lower class" countries in PISA assessment results have led to a research gap. The purpose of this study was to (a) test students' mathematical literacy skills on the Program for International Student Assessment (PISA) test and compare the results using the sum of means across OECD countries; (b) examine the relationship between students' mathematical competence, precision, and self-perception of mathematical literacy skills in the PISA test; and (c) analyze the gaps that exist between the implementation of mathematics instruction in school and the mathematical literacy as measured on the PISA test. This study was designed as a mixed method with an explanatory sequential design. The data collection methods included test procedures, questionnaires, and interviews. The result of this study showed that the overall mean score obtained was below the OECD average. In general, the respondents achieved only level 2 mathematics proficiency. A significant relationship was found between mathematical competence, precision, and self-perception in mathematical skills. On the other hand, there was a gap, namely the difference at the implementation level, where mathematical literacy measured by PISA differed from the measurement of mathematical learning achievement by teachers in school. The results showed that teaching that emphasizes only problem-solving procedures affects low mathematical competence and is not useful enough for students to deal with the PISA mathematics test.</p>

ANALISIS KEMAMPUAN LITERASI NUMERASI DALAM MENYELESAIKAN SOAL PROGRAMME FOR INTERNATIONAL STUDENT ASSESSMENT (PISA)

This study aims to reveal deeper numeracy literacy skills in solving the Program for International Student Assessment (PISA) questions. This research was conducted at SMAS Cerdas Murni Tembung in the odd semester of the 2022/2023 academic year. The research subjects consisted of 28 students of class X MIPA-1. The research uses qualitative methods with a descriptive approach. The research instruments used were PISA test questions and interviews. The results showed that 44% of student were able to fulfill the ability indicators in using syimbols, numbers, and calculate mathematical operations contained in questions at levels 1, 4, and 5. In the indicator of the ability to analyze information from tables, graphs, diagrams, schematics, etc., it was obtained that 57,1% of students were able to solve PISA level 3. As well as indicators of skills in applying concepts as well as making decisions contained in level 2 and level 6 questions, 42,3% of students were able to meet these indicators. So that, the overall numeracy literacy ability of class X MIPA-1 students can be categorized as lacking in solving the Program For International Student Assessment (PISA) questions.

The Validity of Problems Similar to PISA Local Context of North Sumatra Refers to the Components of the PISA Scientific Literacy Framework

The ability of junior high school students in Indonesia is low, especially on the results of the PISA test. PISA (Program for International Student Assessment) is a program to measure achievement for children aged 15 years in the fields of math, science and literacy skills. The results of a survey conducted by the OECD in PISA showed that Indonesia scored 396 out of the OECD average score of 496 on reading skills in the 2012 PISA test. This score placed Indonesia in 64th place. There were 65 countries that participated in the 2009 PISA test. In this regard, it is important to develop questions that can measure these abilities and the ones that are considered most appropriate are questions similar to PISA in the local context of North Sumatra. This study aims to determine the suitability of each PISA-like question in the Local Context of North Sumatra with the PISA framework assessed by the validator. The method used is development research using the ADDIE model, then at the evaluation stage using the Tessmer model.

USING A MACHINE LEARNING APPROACH TO EXPLORE NON-COGNITIVE FACTORS AFFECTING READING, MATHEMATICS, AND SCIENCE LITERACY IN CHINA AND THE UNITED STATES

Non-cognitive factors are considered critical aspects in shaping students' academic achievement. This study aims to analyze and explore the mechanisms of the influence of non-cognitive factors on 15-year-old students' abilities in China and the United States. Based on the Programme for International Student Assessment (PISA) 2018 education dataset, the Classification and Regression Tree (CART) model identifies and explains the factors. The study finds that there are 11 most influential common features in China and 9 in the United States. The two countries have 5 common features, the meta-cognition assess credibility, summarizing text ability, PISA test difficulty perception, science learning time, and school lessons numbers per week. Family economic status also impacts personal ability. Regarding subject characteristics, attitude towards failure is the determinant of reading and mathematics. Cooperation and competition among students help to improve mathematics and science. Furthermore, the comparison between the two countries concludes that self-awareness, family economic status, and school learning environment are critical to personal ability. The study concludes that it is necessary to foster a sense of healthy competition among students at the school level and provide more attention to students with low family socioeconomic status to improve their abilities. Keywords: machine learning approach; non-cognitive factors; PISA 2018; personal ability

Factors and Parameters Influencing Student Achievement in Mathematics: A Comparative Study between Israel and Finland

<p style="text-align: justify;">In all years of the Program for International Student Assessment (PISA) in primary mathematics education, Finland ranked in the lower places (44-61) in the dispersion index (the difference in scores between the 95th percentile and the 5th percentile), while Israel ranked in the upper places (1-3) in the same index. In the last PISA test, Israel ranked first (among the 78 participating countries) in grade differences, while Finland ranked 61st. The score for dispersion in Israel is 356 points, the highest among the countries and economic entities of the Organization for Economic Cooperation and Development (OECD). Based on these results, this comparative study between Israel and Finland was conducted to investigate the most important sociological factor in the Pisa test that influences most student achievements in mathematics in Finland and Israel, as well as the reasons for the differences in achievement between mathematics scores. The results of this study show that the differences in achievement in Israel are due to students' socio-economic status and the sector. In contrast, Finland's first sociological factor influencing student performance is socio-economic status. Nevertheless, it has a more negligible influence than in Israel. The second factor is student motivation.</p>

Pelatihan Penyusunan Instrumen Tes Matematika Online Berbasis PISA Bagi Guru matematika SMP Bengkulu

One of the reference standards in measuring the understanding of secondary school students at the international level is the Program for International Student Assessment (PISA) survey. This is also supported by the implementation of a minimum competency assessment (AKM) in 2021 with a focus on numeracy and literacy skills. However, the results of the PISA survey in 2018 Indonesia only ranked 72 out of 78 participating countries. One of the contributing factors is that students are not accustomed to solving PISA-based questions. So that training on the preparation of PISA-based online mathematics test instruments was carried out. This training activity aims to design a PISA-based online mathematics test instrument. The activity was held at the Bengkulu City Mathematics MGMP which was attended by 45 SMP/MTs mathematics teachers. The implementation of the preparation consists of four stages, namely: (1) module design, (2) question preparation training, (3) structured assignments, (4) online test design. The results of the activity were composed of 25 online PISA test instruments that met the valid criteria. The results of the teacher's response to the training activities on the criteria are good

Global competence in PISA 2018: deconstruction of the measure

ABSTRACT Programme for International Student Assessment (PISA) results have become synonymous with quality benchmarking, determining standards and comparing performance among 15-year-old students in countries around the globe. Concern, however, exists with the utility and consequential validity of the newest measure to the suite of the OECD’s PISA tests: its global competence measure. This study continues our examination of the global competence measure by deconstructing how the OECD frames it within the first global competence rankings, which are based on PISA 2018 data. Our study uses Derrida’s technique of deconstructing texts to critique the five specific claims that the OECD used to underpin the importance of its measures in PISA- policy orientation, innovative concept of literacy, relevance to lifelong learning, regularity, and breadth of coverage. Our analysis exposes the measure’s silences, unquestioned assumptions and inconsistencies. The aporia evident in the text is of concern for educational policy makers around the globe as it perpetuates global messaging that reinforces dominant discourses and continues to ignore contextual differences. We call for continued critique of the global competence measure and recommend that hyperbole be replaced by contextualisation of any differences it might detect.

The impact of test language on PISA scores. New evidence from Wales

AbstractIn this paper we investigate the link between the language in which pupils take the Programme for International Student Assessment (PISA) test and the scores they achieve in this assessment, focusing upon the case of Wales. Using five rounds of PISA data and an instrumental variable approach, we show how pupils who took the test in Welsh score around 0.3 standard deviations (30 PISA test points) lower in reading, mathematics and science than their peers who took the test in English. This finding is robust to different model specifications and statistical approaches. We argue that this may indicate that the academic performance of teenagers in Wales may be underestimated in PISA – particularly amongst those who take this test in Welsh.

An Education Gift — Integrated Cognitive and Non-Cognitive Skills — for Future Generations to Grow the Economy in the Digital Phase

This paper summarizes arguments driving education policy discussion about a relationship between the growth of the economy and early childhood cognitive and non-cognitive skills. The first finding is that rising Harmonized Test Scores, including PISA test scores, do not contribute to labor productivity per person in high income courtiers in Asia, Europe, and North America. On the other hand, the test scores can drive the economy in high income countries in Africa, Caribbean, Middle East, and South America; upper middle-income countries in Africa, Caribbean, Middle East, Ocean, and South America; low middle-income countries; and low middle-income countries more than high-income countries. The second finding is that rising Harmonized Test Scores (HTS) are likely to connect to labor productivity per hour. By a simple regression by taking a logarithm, this study investigates the relationship between labor productivity per hour and the Program for International Student Assessment (PISA) test scores. The coefficient of determination is 0.60. It is not enough to get a sufficient result. Accordingly, the study discusses how labor productivity per hour in high income courtiers in Asia, Europe, and North America is associated with non-cognitive skills. In the digital economic phase, it seems that integrated cognitive skills and non-cognitive skills contribute to labor productivity per hour. We recommend that policymakers should invest in early childhood to not only maintain or improve PISA test scores but also to improve non-cognitive skills associated with psychology. Overall, this paper presents analysis and empirical results, aimed at building a more future-oriented education policy. The audience for this paper includes policymakers, educators, and economists.

The correspondence between PISA performance and school achievement in Finland

In the Programme for International Student Assessment (PISA), the key knowledge and skills needed in modern society are assessed. So far, however, little is known on the relationship between PISA performance and school achievement. Using Finnish PISA 2015 and 2018 data combined with the national register of education records, we studied whether there is correspondence between PISA performance and school achievement. The results showed that the correspondence between PISA performance and school achievement is moderate. The PISA proficiency scores correlated not only with the corresponding grades but also with the grades of other theoretical subjects, indicating that the PISA test assesses a wide range of school achievement. PISA proficiency was related to grades even after controlling for gender and SES.

Secondary school students’ English literacy achievement based on PISA reading literacy test 2009

Reading literacy has become global concern that Organisation for Economic Co-operation and Development (OECD) has conducted Program for International Student Assessment (PISA) reading literacy test to assess what students know and can do in reading. Participating in PISA test since 2000, Indonesian students’ reading literacy has not shown a significant improvement. Because PISA reading literacy test is adapted into language of instruction of each participating country, it is Indonesian students’ reading literacy in Indonesian language which is not satisfactory. Considering the fact that Indonesian fifteen-year-old students study English as a compulsory subject, investigating students’ reading literacy in English is considered important. There has been much research investigating issues in reading achievement in Indonesia, but there are limited studies focussing on PISA reading literacy based on school location, school accreditation, gender, and academic major. Employing survey study design, this study assessed reading literacy of two thousand and two hundred secondary school students sampled from twenty four public secondary schools accredited A+, A, and B in Palembang. Data were obtained using PISA reading literacy test 2009 and were analysed statistically. The findings revealed that students’ English literacy was in level 3 (low category). Students studied at schools in central urban district performed better than those of in peri urban district. Those from secondary schools accredited A+ outperformed their counterparts in schools accredited A and B. Female students performed slightly better than male. Students majoring in science were better than those of majoring in social. The results lead to the conclusion that students’ English literacy achievement should be enhanced using innovating strategies to achieve targeted literacy level that is needed to survive in school academic life.

The Impact of Integrating Critical Thinking Skills in Reading Curriculum in Developing Qatari Students Performance in PISA Exams

This study aims at identifying the effect of integrating critical thinking skills in the Arabic language curriculum for the seventh grade in the State of Qatar through the reading course on improving the performance of Qatari students in the Programme for International Student Assessment (PISA) tests, and the study used the descriptive-analytical method. It analyzed the content of the reading course for grade seven in Qatar and the content of the PISA test in the field of reading knowledge and their critical thinking skills. The results of the study showed a clear improvement in the performance of Qatar’s students in the Pisa tests in general, as Qatar’s students made remarkable progress between 2006 and 2015, where the general average in -the field of reading literacy -increased from (312) to (402), which indicates - according to the study - the effect of integrating critical thinking skills in achieving this.

How Does Education Quality Affect Economic Growth?

In a seminal article, Hanushek and Woessmann explained economic growth as a function of the quality of education. While they did not find evidence of the importance of years of schooling, they argued for the relevance of cognitive skills and a basic literacy ratio for economic growth. However, this result was based on cross-country data limited to 23 observations. In this study, we extended and modified their approach based on the results of PISA (Programme for International Student Assessment) tests to explain the GDP changes over the last 50 years. Using panel data, we considered the possible lag that characterizes this relationship, used statistical methods to address the risk of reversed causality of economic performance affecting the quality of education, and extended the model by the inclusion of other potential growth factors. The results, which also included several robustness checks, confirmed the relevance of earlier education quality as a significant growth factor. Our results suggest the significance of educational skills for GDP growth, which might be treated as a confirmation of the importance of quality primary and secondary education for economic development. We showed that our results are robust to changes in the order of lags and confirmed the validity of the conclusion with the use of specification-robust Bayesian model averaging.

A comparative study on cognitive diagnostic assessment of mathematical key competencies and learning trajectories

Initiated by the Program for International Student Assessment (PISA), Mathematical Key Competencies (MKCs), which integrate into the process of solving situational problems, becomes a typical example of Mathematical Key Competencies assessment. Cognitive Diagnostic Assessment, as a new generation of measurement theory, integrates the measurement objectives into the cognitive process model via cognitive analysis, and helps increase understanding of students’ mastery over fine-grained knowledge points. This paper analyzed 12 PISA test items and calibrated their attributes, thus forming a cognitive model based on six PISA’s MKCs, which were namely Mathematical Abstraction, Logical Reasoning, Intuitive Imagination, Mathematical Modeling, Mathematical Operation and Data Analysis. Through the comparison of models fit for DINA, DINO, RRUM, ACDM, GDM, LCDM, LLM, G-DINA and Mixed Model, the LCDM with a good model fit was selected to analyze the data of 19, 454 students in eight countries, and comparisons of the six MKCs among these countries were obtained. Through analyzing the knowledge states, combing the prerequisite relationships between the attributes, and exploring the learning trajectories of students’ MKCs in different countries, we found that the students’ performance in China was the best among all countries for each of the six MKCs. In all other countries, the students’ performance in Logical Reasoning and Intuitive Imagination were weaker than the other attributes. An analysis of learning trajectories found that Russia, Singapore, Australia and Finland had very similar main learning trajectories (highlighted in red), and the main learning trajectories of Russia and Singapore are the same; however, there exist distinct differences in learning trajectories between China and the United States, where the learning trajectories in China is complicated with varied branches, while the ones in the United States is relatively simple with fewer branches.

Metacognitive Monitoring and Mathematical Abilities: Cognitive Diagnostic Model and Signal Detection Theory Approach

Besides various in-class assessments, there exist some standardized assessment tools that are administered in several countries, such as PISA (Programme for International Student Assessment) and TIMMS (Trends in International Mathematics and Science Study). The questions’ contents, type of responding, grading, and the analyses in these large-scale tests have been diversified in years. In this study, it was aimed to identify the abilities that are measured at PISA mathematics test in a single testing procedure and by utilizing the methods of analyses of Cognitive Diagnostic Model (CDM) as well as Signal Detection Theory (SDT), which have not been used so far in the assessment of these abilities. Therefore, a randomly selected sample of 6th-grade students ( N =230) in Izmir was tested with a PISA-equivalent 12-item mathematics test, where the items are graded dichotomously (correct vs. incorrect). CDM estimates were calculated by using the Deterministic Input Noisy Output and Gate (DINA) Model. The participants were asked to report whether they thought they could solve the question correctly, guess even if they thought they could not solve the question, and then, rate their confidence levels on the correctness of their answers in turn so as to allow us to measure their “metacognitive monitoring performance” with the SDT method, which refers to the ability to differentiate correct and incorrect responses. In short, a better metacognitive monitoring performance was obtained by measuring how well once could differentiate their correct and incorrect responses with the observation of they prefer reporting and then giving high confidence levels to the actually correct responses and prefer passing to give an answer yet rate lower confidence levels to the actually incorrect responses given as pure guesses. The results showed that CDM fits well to the assessment of PISA test and those who were better at the ability of “reasoning and developing strategies” in particular among four possible abilities detected with CDM (“representing and communicating”, “mathematization”, “reasoning and developing strategies”, “using symbolic and technical language”) had also better metacognitive monitoring performance. The present study, therefore, contributes to the research that investigates what features the ability of better differentiating correct and incorrect responses are actually linked. Based on the results, it is suggested that a better metacognitive monitoring ability is linked to having a better ability of “reasoning and developing strategies” in particular. Additionally, it is suggested that measuring metacognitive monitoring performance at PISA -or even any other possible tests- with the SDT calculation method, that has a relatively straightforward testing procedure, may yield various estimates for the students’ abilities measured at the test as well as their related higher-order abilities.

Early school entrance and middle-run academic performance in Mexico: evidence for 15-year-old students from the PISA test

Using an unanticipated policy reform in Mexico that shifted 4 months the cutoff date for elementary school eligibility in 2006 and information on academic performance of 15-year-old students from the Programme for International Student Assessment (PISA) 2018 survey, who entered first grade in 2006, this paper estimates the effects of starting first grade without having reached 6 years of age on middle-run academic performance. We found that entering school at a younger age increases the probability of having repeated at least one academic year by 7 percentage points and reduces academic performance (measured throughout PISA’s score points) between 6 and 10 score points in mathematics, reading and science (a Cohen’s d effect size between 0.06 and 0.10). We did not find evidence to claim that the effect of early entrance on academic performance is different for girls and boys or for urban and rural locations.

International Comparative Study on PISA Mathematics Achievement Test Based on Cognitive Diagnostic Models

As one of the most influential international large-scale educational assessments, the Program for International Student Assessment (PISA) provides a valuable platform for the horizontal comparisons and references of international education. The cognitive diagnostic model, a newly generated evaluation theory, can integrate measurement goals into the cognitive process model through cognitive analysis, which provides a better understanding of the mastery of students of fine-grained knowledge points. On the basis of the mathematical measurement framework of PISA 2012, 11 attributes have been formed from three dimensions in this study. Twelve test items with item responses from 24,512 students from 10 countries participated in answering were selected, and the analyses were divided into several steps. First, the relationships between the 11 attributes and the 12 test items were classified to form a Q matrix. Second, the cognitive model of the PISA mathematics test was established. The liner logistic model (LLM) with better model fit was selected as the parameter evaluation model through model comparisons. By analyzing the knowledge states of these countries and the prerequisite relations among the attributes, this study explored the different learning trajectories of students in the content field. The result showed that students from Australia, Canada, the United Kingdom, and Russia shared similar main learning trajectories, while Finland and Japan were consistent with their main learning trajectories. The primary learning trajectories of the United States and China were the same. Furthermore, the learning trajectory for Singapore was the most complicated, as it showed a diverse learning process, whereas the trajectory in the United States and Saudi Arabia was relatively simple. This study concluded the differences of the mastery of students of the 11 cognitive attributes from the three dimensions of content, process, and context across the 10 countries, which provided a reference for further understanding of the PISA test results in other countries and shed some evidence for a deeper understanding of the strengths and weaknesses of mathematics education in various countries.

Is Canada really an education superpower? The impact of non-participation on results from PISA 2015

The purpose of large-scale international assessments is to compare educational achievement across countries. For such cross-national comparisons to be meaningful, the participating students must be representative of the target population. In this paper, we consider whether this is the case for Canada, a country widely recognised as high performing in the Programme for International Student Assessment (PISA). Our analysis illustrates how the PISA 2015 sample for Canada only covers around half of the 15-year-old population, compared to over 90% in countries like Finland, Estonia, Japan and South Korea. We discuss how this emerges from differences in how children with special educational needs are defined and rules for their inclusion in the study, variation in school participation rates and the comparatively high rates of pupils’ absence in Canada during the PISA study. The paper concludes by investigating how Canada’s PISA 2015 rank would change under different assumptions about how the non-participating students would have performed were they to have taken the PISA test.